Skip to content

Cache occupancy in WorkStealing.balance()#9005

Merged
hendrikmakait merged 2 commits intodask:mainfrom
hendrikmakait:cached-occupancy-in-balancing
Feb 5, 2025
Merged

Cache occupancy in WorkStealing.balance()#9005
hendrikmakait merged 2 commits intodask:mainfrom
hendrikmakait:cached-occupancy-in-balancing

Conversation

@hendrikmakait
Copy link
Member

@hendrikmakait hendrikmakait commented Feb 5, 2025 •
edited
Loading

Calculating the occupancy is somewhat expensive, so we should avoid it if possible.

  • Tests added / passed
  • Passes pre-commit run --all-files

@github-actions
Copy link
Contributor

github-actions bot commented Feb 5, 2025

Unit Test Results

See test report for an extended history of previous test failures. This is useful for diagnosing flaky tests.

    27 files  +  1      27 suites  +1   11h 31m 44s ⏱️ + 40m 42s
 4 118 tests +  7   4 004 ✅ + 11    111 💤  -  2  3 ❌  - 1 
51 641 runs  +419  49 340 ✅ +363  2 297 💤 +57  4 ❌ ±0 

For more details on these failures, see this check.

Results for commit 6917e8b. ± Comparison against base commit b86b714.

This pull request removes 2 and adds 9 tests. Note that renamed tests count towards both. 
distributed.cli.tests.test_dask_worker.test_listen_address_ipv6[tcp:..[ ‑ 1]:---nanny]
distributed.cli.tests.test_dask_worker.test_listen_address_ipv6[tcp:..[ ‑ 1]:---no-nanny]

distributed.diagnostics.tests.test_memray ‑ test_basic_integration_scheduler
distributed.diagnostics.tests.test_memray ‑ test_basic_integration_scheduler_report_args[False]
distributed.diagnostics.tests.test_memray ‑ test_basic_integration_scheduler_report_args[report_args0]
distributed.diagnostics.tests.test_memray ‑ test_basic_integration_workers[1]
distributed.diagnostics.tests.test_memray ‑ test_basic_integration_workers[False]
distributed.diagnostics.tests.test_memray ‑ test_basic_integration_workers[True]
distributed.diagnostics.tests.test_memray ‑ test_basic_integration_workers_report_args[False]
distributed.diagnostics.tests.test_memray ‑ test_basic_integration_workers_report_args[report_args0]
distributed.tests.test_steal ‑ test_stealing_objective_accounts_for_in_flight

@hendrikmakait hendrikmakait force-pushed the cached-occupancy-in-balancing branch from 6917e8b to e4d3730 Compare February 5, 2025 11:55
@hendrikmakait hendrikmakait force-pushed the cached-occupancy-in-balancing branch from e4d3730 to bc77216 Compare February 5, 2025 11:58
@hendrikmakait hendrikmakait marked this pull request as ready for review February 5, 2025 11:58
@hendrikmakait hendrikmakait changed the title Cache occupancy in balancing Cache occupancy in WorkStealing.balance() Feb 5, 2025
@hendrikmakait hendrikmakait merged commit e5711c6 into dask:main Feb 5, 2025
29 of 32 checks passed
@hendrikmakait hendrikmakait deleted the cached-occupancy-in-balancing branch February 5, 2025 12:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@phofl phofl phofl approved these changes

@fjetter fjetter Awaiting requested review from fjetter fjetter is a code owner

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@hendrikmakait @phofl